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Abstract 

This paper is concerned with linear stochastic control systems in state space. The integral of the squared 
norm of the system output over a bounded time interval is interpreted as energy. The cumulants of the output 
^j", energy in the infinite-horizon limit are related to Schatten norms of the system in the Hardy space of transfer 

■ functions and the risk-sensitive performance index. We employ a novel performance criterion which seeks to 
minimize a combination of the average value and the variance of the output energy of the system per unit time. 

■ The resulting linear quadro-quartic Gaussian control problem involves the H2 and "T^-norms of the closed-loop 
(3JT)[ system. We obtain equations for the optimal controller and outline a homotopy method which reduces the solution 

of the problem to the numerical integration of a differential equation initialized by the standard linear quadratic 
Gaussian controller. 

^ ■ I. Introduction 

■ 

This paper is concerned with linear multi-input multi-output control systems, governed in state space 

■ by Ito stochastic differential equations, driven by a standard Wiener process which is regarded as a 
C/} random disturbance. The integral of the squared Euclidean norm of the system output over a bounded 

c/j ! time interval is interpreted as energy. In the disturbance attenuation paradigm, the output energy is to 
, ^ ' be minimized in some sense. 

Linear Quadratic Gaussian (LQG) control [1], for example, seeks to minimize the expectation of the 
^ . output energy which, in the infinite-horizon limit, reduces to the squared "H 2 -norm of the closed-loop 
lO system in an appropriate Hardy space of transfer functions. An alternative performance index is employed 
in the Risk-Sensitive and Minimum Entropy control theories [11]. They utilise the expected value of 
^ the exponential of the output energy multiplied by a scaling parameter to adjust the risk sensitivity. 
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Risk-sensitive control extends the LQG approach and is robust with respect to Kullback-Leibler relative 
O ! entropy bounded uncertainties in the random noise [2]. 

The risk-sensitive performance index can be represented as a series expansion with respect to the 
energy scaling parameter. The coefficients of this series are the rates of the asymptotically linear growth 
. £h ! of the cumulants of the output energy in the infinite-horizon limit. The cumulant growth rates are directly 
related to higher-order Schatten norms [12] of the transfer function of the system in an appropriate 
Hardy space. This allows the risk-sensitive criterion to be viewed as a linear combination of powers 
of Hardy-Schatten norms of the system whose weights are governed by the risk-sensitivity parameter 
in a very specific way. The "reverse engineering" of the risk-sensitive index suggests a wide family of 
performance criteria in the form of linear combinations of powers of the Hardy-Schatten norms. This 
gives rise to a class of output energy cumulant (OEC) control problems which extend the risk-sensitive 
paradigm. In fact, the LQG approach can be considered to explore this freedom to a certain degree by 
retaining the first term (the squared "H 2 -norm of the system) of the risk-sensitive index expansion. 

The present paper develops the OEC control idea, outlined above, by employing a performance 
criterion which seeks to minimize a combination of the average value and the variance of the output 
energy of the system per unit time. The resulting linear quadro-quartic Gaussian (LQQG) control problem 
utilizes a quadro-quartic functional as a finer truncation of the risk-sensitive performance index which 
retains the T-L 2 an d "H/i-norms of the closed-loop system and the risk-sensitive parameter. 
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igor . g . vladimirov@gmail . com, i . r . petersenSgmail . com. 



The H4-norm, which involves the Schatten 4-norm of matrices [6] and is referred to as the quartic 
norm, was introduced in [13] as a subsidiary construct in the anisotropy-based robust control theory 
for discrete-time stochastic systems. In the present study, the quartic norm plays a central role and, 
in addition to providing the next term in the risk-sensitive index expansion, quantifies (via the H± to 
H2-norms ratio) the time scale beyond which the infinite-horizon LQG cost starts manifesting itself in 
sample paths of the output energy of the system. 

We consider the LQQG problem in the class of linear stabilizing controllers with the same state 
dimension as the underlying plant. This allows equations for an optimal controller to be obtained by 
using Frechet derivatives of the quadro-quartic performance index of the closed-loop system with respect 
to the state-space realization matrices of the controller. The resulting set of equations depends on the 
risk sensitivity parameter and yields the standard LQG controller for a zero value of the parameter. 
We outline a homotopy method which regards the parameter as a fictitious time variable and reduces 
the solution of the set of equations to a problem involving the numerical integration of an ordinary 
differential equation (ODE) initialized by the standard LQG controller. 

In addition to its possible extension to the discrete-time case, the LQQG approach may also find 
application in the control of quantum stochastic systems as an alternative to the risk-sensitive control 
paradigm. 

II. Variance of output energy and quartic norm 

Suppose W := (w t )tm is a m -dimensional standard Wiener process (initialised in the infinitely 
distant past) at the input of a linear time invariant (LTI) system F with a square integrable M pxm -valued 
impulse response function / := (ft)t^o> see Fig- HI The output Z := (z t )tm of the system is a Revalued 
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Fig. 1. An LTI system F with input W and output Z. 



Gaussian random process defined by the Ito stochastic integral z t := f_ f t - s dw s . The mean value of 
Z is zero and the covariance function is 



c t := E(z t z^) = / f s+t fjds = — / S(u)e^du = c T _ t , t > 0, 

J0 ^ J-oo 



(1) 



where 

S(u) := F(u)F(u)* = / c t e~ iult dt (2) 



is the spectral density of Z. Here, (■)* := ((-)) T denotes the complex conjugate transpose of a matrix, 
and F(u) := F(iu) = J +o ° fte~ luJt dt is the Fourier transform of the impulse response, that is, the 
boundary value of the transfer function of the system F(v) := / + °° f t e~ vt dt, with Rev > 0. With / 
assumed to be square integrable, F belongs to the Hardy space 'H^*™ of C pxm -valued functions of a 
complex variable, analytic in the right half-plane and endowed with the T^-norm 
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WftW 2dt =J^ |^||iWdu, (3) 



Here, the Plancherel theorem is used, and ||M|| := a/ (M, M) denotes the Frobenius norm of a matrix 
M generated by the inner product (M, N) := Tr(M*N), so that ||-F(w)|| 2 = TvS(u) is the trace of the 
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spectral density from ©. In view of ©, ||-F|| 2 = Trc = E ( | ^ | 2 ) is the variance of the output signal 
for any t. For a finite time horizon T > 0, the random variable 

£ T := I \z t \ 2 dt (4) 



Jo 

is interpreted as the output energy of the system F over the time interval [0, T], and 

e T := £ T /T (5) 

is the corresponding output energy rate. The mean value of coincides with the squared T^-norm of 
the system ©: Ee^ = \\F\\%. This ensemble average can manifest itself in sample paths of ey only 
by virtue of the law of large numbers, provided T is large enough. Under additional assumptions on 
the system F, the rate of the mean square convergence l.i.m. 7^+00 ex = \\F\\l is quantified by the 
asymptotic behaviour of the variance of e T . The convergence rate is described by the lemma below in 
terms of the quantity 
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This is a continuous-time counterpart of the "H 4 -norm introduced as a subsidiary construct in the 
anisotropy-based robust control of discrete-time systems [13]. The second equality in © follows from 
the Plancherel theorem applied to the spectral density ©. The systems F with \\F\\ A < +oo form a 
normed space U p 4 xm . The integrand ||5(w)jj 2 = Tr((F(w)F(u;)*) 2 ) in © is the fourth power of the 
Schatten 4-norm [6, p. 441] of the matrix F(u>); see also [12]. The H 4 -norm ||F|| 4 will be referred to 
as the quartic norm of the system F. 

Lemma 1: Let F E X ™ D ^4 Xm - Then the variance of the output energy rate © of the system 
behaves asymptotically as 

var(e r ) ~ 2\\F\\j/T, T -> +oo. (7) 
Proof: By applying Lemma [6] of Appendix A to the Gaussian random vectors z s and z t and using 
©, it follows that cov(|2; s | 2 , \z t \ 2 ) = 2||c s _ t || 2 . Hence, the variance of the output energy © can be 
computed as 

var(fir) = / cov(|z s | 2 , \z t \ 2 )dsdt = 2 I ||c s _ t || 2 dsdt = 4T I (1 - u/T) ||c u || 2 du, (8) 

J[0,T] 2 J[0,T] 2 JO 

where use is made of the property c t = c^ t and the invariance of the Frobenius norm of a matrix 
under the transpose. Since the assumption F E "H^" 1 ensures the square integrability of the covariance 
function ©, then 
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lim / (l-u/T)||cJ| 2 du= / \\c u \\ 2 du (9) 

T— >+oo 



holds by Lebesgue's dominated convergence theorem. Since dr / Q T (1 _ U /T) ||c u || 2 du = 
T~ 2 J T M||c u || 2 dM ^ 0, the convergence is monotonic. Now, © is obtained by using © and combining 
© and © with ©: var(e r ) = var(£ T )/T 2 ~ 4 f+°° ||c u || 2 dM/T = 2||F|||/T as T -> +oo. ■ 
In view of a central limit theorem for quadratic functionals of Gaussian processes [4, Theorem 2], 
the relation © provides the scaling factor for the asymptotic standard normality of the random variable 
sjF '/2(er — 1 1 F\\ |)/ 1 1 F\\ \ as T — > +oo. Heuristically, the root mean square deviation of er from its 
mean value \\F\\l is relatively small if 



T»T* :=2(||F|| 4 /||F|| 2 ) 4 . (10) 
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The right-hand side of ( flOl ) quantifies the time horizon beyond which the T^-norm ||F|| 2 manifests itself 
in the sample paths of the output energy of the system. On the other hand, for T <^ T*, the ergodic 
properties of the system output Z do not expose themselves since the expected value Ee T = ||F||| of 
the output energy rate is "indistinguishable" in the background of random fluctuations whose standard 
deviation can be estimated by using © as A/var(e^) ~ || y/TJT > Ee T . Thus, the squared H 2 - 
norm as the average output energy loses its significance for quantifying the disturbance attenuation 
capabilities of the system on short time scales T -C T». The critical time horizon T* defined by (flOl) is 
similar to the integral time scale of measurements in turbulent flows [3, pp. 50-51]. As an example, let 
Z be an Ornstein-Uhlenbeck process generated from a standard Wiener process If by a single-input 
single-output system F according to the SDE 

dzt = az t dt + y/2\a\dwt, (11) 

parameterized by a < 0. The covariance function (OQ) of Z is c t = e a '*l, and the H2 and "H 4 -norms of 
the system F, defined by © and ©, are ||F|| 2 = 1 and ||F|| 4 = la)" 1 / 4 . Therefore, the critical time 
horizon (flOl ) takes the form = 2/|a| and coincides with the typical transient time of the process; see 

Fig.m 




Fig. 2. 100 sample paths of er versus T ^ 10 for the Ornstein-Uhlenbeck process generated by i ll It with a = — 1, so that the critical 
time horizon beyond which et exposes relative proximity to the limit value ||-F||2 = 1 (horizontal bold line) is T» = 2. The dashed bold 
lines localize the typical values of er which form a "tube" of half-width ^/T*/T about the limit. 



III. CUMULANTS OF OUTPUT ENERGY AND HARDY- SCHATTEN NORMS 

For a finite time horizon T > 0, let Ct denote a Toeplitz integral operator whose kernel is specified 
by the covariance function (OQ). An W- valued integrable function ip := ( , 0t)o<t<T is mapped by Ct to 
tp := (<^ s )o^s^t as Lp s := J Q T c s _ t ^ 4 dt. Suppose 6 is a real parameter satisfying < 6 < l/p(C T ), where 
/?(•) is the spectral radius. In view of the Fredholm formula [12, Theorem 3.10 on p. 36] (see also [4] 
and references therein), 

lnEe^ r/2 = --Trln(J - 9C T ) = -^0 fe Tr(C£)/A;, (12) 
where I is the identity operator. The trace of the A;-fold iterate of Ct is computed as 

Tt(Ct) = / T^Cio-tiCti-ia X • • • X Ctfc-a-tfc-iCtfc-i-fo)^ X ... X dt k ^. (13) 
J[0,T] k 
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The expectation in (fT2l) is the moment-generating function of £t, and hence, 

lnEe^ T/2 = ^(0/2) fc K fe (£ T )/A;! = 9 {ES T + #var(£ T )/4) /2 + 0(6 3 ), 9^0. (14) 



Here, K fc (£) := ^lnEe^| = P k (E^, . . . , E(£ fc )) denotes the fcth cumulant of a random variable 
£, which is related with the first fc moments of £ via a universal polynomial P k . The first three of 
these polynomials are Pi(/ii) = fii, P 2 (/ii,/i 2 ) = A«2 - lA. and ^3(^1,^2,^3) = A*3 - 3/ii/x 2 + 2/zf. By 
comparing the power series in (PT21) and (PT4T) and using the identity (2r)H = r\2 r , it follows that the fcth 
cumulant of the output energy © of the system is related to the trace (TT3l) as 

K k {£ T ) = {2k-2)\m{C T ). (15) 

Using © and extending ® and ©, we define, for a positive integer k, a higher order Hardy norm of 
the system F by 



\2k 




Tr(S(u) k )du, (16) 



which reproduces the H 2 an d T^-norms for k = 1, 2. Here, 2 ^/Tr(S(u;) fc ) is the Schatten 2/c-norm [6, 
p. 441] of the matrix P(w). The resulting Hardy-Schatten space H^k m ls equipped with the norm || • || 2fc . 
Similarly to the "H 2 -norm, the "H 2fc -norms ([TBI are all invariant under replacing the system F with its 
dual F\ 

\\F% k = \\F\\ 2k , k^l, (17) 

where F^ has the transposed impulse response (fj)t^o- Indeed, the transpose of a square matrix does 
not modify its spectrum, and for conformable complex matrices X and Y, the matrices XY and YX 
share nonzero eigenvalues. Therefore, with the dependence on the frequency to omitted for brevity, 
Tr((F T (F T )*) fc ) = Tr(((F*F) fc ) T ) = Tr((FF*) fc ), and hence Q3 follows. By the Szego limit theorem 
for Toeplitz operators [5], under additional integrability conditions, 

lim = — / Trx (S (u ))du. (18) 

Here, x is a function of a complex variable, satisfying x(0) = and analytic in a neighbourhood 
of the interval [0, ||P||^o], with ||P||oo the %oo-norm of F. In view of (fT5l) . the application of (fl~8l 
to elementary polynomials x( v ) '■= v k yields the asymptotically linear growth of the output energy 
cumulants with respect to time: lim T _ >+00 (K fc (£ T )/T) = (2k - 2)!!||F|||£, provided F e fljLi^l?™ 
with Lemma [TJ being a particular case for k = 2. The application of ([TBI to x{ v ) '■= (2/0)ln(l — 9v), 
with < 9 < ||P||~ 2 , gives 

2 ,. lnEe^ 2 1 f +co , , fT nn . 

- lim — = / lndet(J„ — 9b{uj))a.uj 

t^+oc T 2ti9 7.^ v p v 11 

= J2 dk ~ l W F W%/ k = ^(F) + 0(9 2 ), 9 0+, (19) 
fe>i 

where J p denotes the identity matrix of order p, and 

Qe{F) := ||F|| 2 + #||F||4/2. (20) 

The expected exponential-of-quadratic functional Ee e ^ T / 2 in (fT9~l) is used as a performance criterion 
in the risk-sensitive and minimum entropy control theories [11]. The quartic norm \\F\\ 4 provides the 
next correction to the squared 'H 2 -norm in the series expansion (fT9~l) for small 9. Therefore, the 

quadro-quartic functional Qq, defined by (1201) . can be regarded as a finer truncation of the risk-sensitive 
performance index. 
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IV. QUADRO-QUARTIC FUNCTIONAL IN STATE SPACE 

Let F be a strictly proper LTI system with an m-dimensional standard Wiener process W at the 
input, p-dimensional output Z and n-dimensional state X governed by an Ito SDE: 

dx t = Ax t dt + Bdw t , z t = Cx t , (21) 

where A E R raxrt , B E ]R nxm , C E W xn are constant matrices. The state-space representation will be 
written as 



F = (A,B,C) = 



4 


-n— >< 


— m— > 




' A 


B ' 


V X 


C 






where we have also shown the dimensions, and the horizontal and vertical separators serve to avoid 
confusion with an ordinary block matrix. The dual system is F^ = (A T ,C T , B T ). If the matrix A is 
Hurwitz, then the mutually dual controllability and observability Gramians P and Q of (1221) are unique 
solutions of the algebraic Lyapunov equations 

AP + PA T + BB T = 0, A T Q + QA + C T C = 0. (23) 

In what follows, an important role is played by the matrix 

H := QP, (24) 

whose spectrum is formed by the squared Hankel singular values of the system (l22l) . We will write 
1 1 -X" 1 1 a/ := \/ r T 1 c(X T MX) for the weighted Frobenius (semi-) norm of a real matrix X generated by a 
positive (semi-) definite matrix M. 

Lemma 2: Let F be an asymptotically stable system with the state-space realization d22l) . Then the 
quartic norm © is expressed in terms of the Gramians P, Q from (1231 and the matrix H from ([24]) as 

\\F\\i = 2\\(A,PC T ,C)\\l = 2\\PC T \\ 2 9 = 2\\(A,B,B T Q)\\l = 2\\QBf P = —ATi{A T H 2 ). (25) 
Proof: Let Z be a stationary Gaussian random process generated by (T2T)) . with W a standard 
Wiener process. Then the steady-state covariance function © is 

Ct = Ce M PC T , t^O. (26) 

Here, we use the fact that the controllability Gramian is the steady-state covariance matrix of the state 
of the system: P = cov(x t ). Since the function c t in (l26l) coincides with the impulse response of the 
system {A,PC T ,C), then © yields \\F\\j = 2\\{A, PC T , C)\\\ = 2Ti{CPQPC T ) = 2\\PC T \\ 2 Q , which 
proves the first two equalities in (1231) . Here, we have also used the property that the system (A, PC T , C) 
shares the matrices A, C with the underlying system (1221) and hence, inherits from F the observability 
Gramian Q. The remaining three equalities in (|25T) follow from the first two by the invariance of the 
H2 and %4-norms under taking the dual of a system, and by the duality of the controllability and 
observability Gramians. ■ 
The controllability and observability Gramians $, \& of a subsidiary system (A, PC T , B T Q), which 
satisfy the algebraic Lyapunov equations 

A$ + $A T + PC T CP = 0, A T ^ + <&A + QBB T Q = 0, (27) 

will be referred to as the controllability and observability Schattenians of the system d22l) . The 
representations ( 1231 ) imply that 

\\F\\\ = 2Tr(C$C T ) = 2Tr( J B T W), 
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A 5, 



and hence, the significance of the Schattenians $, ^ for the quartic norm is analogous to the role which 
the Gramians P, Q play for the T^-norm. 

Theorem 1: Let F be an asymptotically stable system with the state-space realization (|22)) . Then the 
quadro-quartic functional (1201 ) is expressed in terms of the Gramians P, Q from (1231 and the matrix H 
from (Ell) as 

Q,(F) = ||(4[B \/0PC7 T ],C7)||* 
= Tr((£?£? T + 6PC T CP)Q) 
C 

V6B T Q_ 

= Tr((C T C + 6QBB T Q)P) = -2Tr(A T H(I n + OH)). (28) 
Proo// Substitution of ||F|| 2 = y/Tr (B T QB) and the first two equalities from (O into (gO]) yields 

= || (A [B V9PC T ] ,C)\\l = Ty((BB t + 6PC T CP)Q), 

which establishes the first two equalities in (l28l ). The third and fourth representations of the quadro- 
quartic functional are obtained from the first two by the duality argument or directly from the third and 
fourth equalities in (l25l) . The last representation of Qe(F) m (EH) follows from the previous ones by 
using the Lyapunov equations (l23l) : 



Q (F) = Tr((BB T + 9PC T CP)Q) 

= -Tr((AP + PA T + 6P(A T Q 
= -2Tt(A t QP + 8A T (QP) 2 ) = 



QA)P)Q) 

-2Tr(A T H(I n + OH)). 



V. Linear quadro-quartic Gaussian control problem 

Consider a plant with an mi-dimensional standard Wiener process W as the input disturbance and an 
m 2 -dimensional input control signal U . The outputs of the system are a pi -dimensional to-be-controlled 
signal Z and a p 2 -dimensional observation signal Y. Also, the system has an n-dimensional state X. 
These processes are governed by 



Here, A e 



dx t 

z t 

dyt 

\ B k e R nxmk , Cj e 



Ax t dt + Bidwt + B 2 u t dt, 
Cix t + D u u t , 
C 2 x t dt + D 2 idw t . 



(29) 
(30) 
(31) 



D jk e RPi xm \ with D u = and D 22 = 0. The control 



signal U is generated at the output of a controller K with input Y. We consider a strictly proper LTI 
controller 



-vi- 



K 



n X 


a 


b ' 


m.2 X 


c 






(32) 



with an n-dimensional state S. It is driven by the observation Y and produces the output U as 

d& = a£t<tt + bdy t , u t = cf t , (33) 
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where a G 



\ b G W nxp \ c G R m2Xn . The closed-loop system 

<— 2n— > <— mi— > 



2n t 


A 


B 


Pi t 


C 






a 


6C 2 


6^21 ' 


B 2 c 














(34) 



governed by (|29l)-(l33l) and depicted in Fig. [3l has the 2n-dimensional combined state (E,X). We 

Z 



Y 





plant 










K 







w 

u 



Fig. 3. The closed-loop system F with input W and output Z. 



formulate a linear quadro-quartic Gaussian (LQQG) control problem as the minimization of the 
functional (1201) over n-dimensional controllers (l32b such that the matrix of the closed-loop system 
in (1341) is Hurwitz: 



Q := Q e (F) = -2Tt(A T H(I 2n + OH)) — y min, K stabilizes F. 



(35) 



Here, 9 ^ is a given parameter as before, and use is made of Theorem Q3 so that the matrix H is 
associated by (T24l) with the Gramians P, Q of the closed-loop system satisfying the algebraic Lyapunov 
equations 



AP + PA T + BE 



0. 



A L Q + QA + C L C = 0. 



(36) 



In the case = 0, the LQQG problem (1351) reduces to the standard linear quadratic Gaussian (LQG) 
control problem. For 9 > 0, the LQQG problem is a compromise between minimizing the mean value 
and the variance of the output energy per unit time, with 9 becoming the relative weight of the quartic 
norm. 

VI. Matrices with T-shaped sparsity 

Since it is convenient to assemble the state-space realization matrices into a matrix with 'T-shaped" 
sparsity, we denote the set of real (r + p) x (r + m) -matrices with zero bottom-right block of size 
(p x m) by 



r,m,p • 



P a 
t 



: p G M rxr , a G M rxm , r G 



ppxr 



(37) 



This is a linear subspace of ]^( r +p) x ( r + m ) which inherits the Frobenius inner product of matrices. Let 
Ilrm n denote the orthogonal projection onto T r>mp which pads the bottom-right (p x m) -block of a 



r,m,p 



(r + p) x (r + m) -matrix with zeros: 



n 



r,m,p 



( 


P <r 


)- 


P 


a 




T W 




T 






(38) 



The dependence of the closed-loop system matrices A, B, C on the controller matrices a, b, c in y4 
can be written as 



A B 
C 



r + Ti7r 2 , 



7 - 



a b 
c 



(39) 
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The affine map T n>P2tm2 3 7 t-+ T G r 2 „ imiiPl is specified completely by three matrices 



n 
A B l 
Cx 



In 
P 2 

P 12 



n 





C*2 D21 



(40) 



where 0„ denotes the (n x n)-matrix of zeros. 



VII. Equations for optimal controller 

We now obtain necessary conditions of optimality in the class (l32l) of n-dimensional stabilizing 
controllers K for the LQQG problem (l35l) . To this end, we compute the Frechet derivatives of the 
quadro-quartic functional of the closed-loop system F as a composite function 7 H- V H- Q of the 
controller matrices a, b, c and equate the derivatives to zero. The differentiation is carried out in two 
steps: we first consider A, B, C to be independent variables, and then take into account their dependence 
on a, b, c. 

Lemma 3: The Frechet derivatives of the quadro-quartic functional Q with respect to the closed-loop 
system matrices A, B, C, assembled into the matrix T in (|39l) , are computed as 



d r Q := 







R VLB 
CT 



(41) 



Here, 



T 

n 

R 



p- 
Q- 

H 



6(PH + $), (42) 
9{HQ + m), (43) 
-6(H 2 + Q$ + ^P), (44) 

with P, Q the Gramians from (l36l) ; the matrix H is given by (1241) . an d ^ are the controllability and 
observability Schattenians of F satisfying the algebraic Lyapunov equations 

A§ + + PC r CP = 0, A T V + VA + QBB T Q = 0. (45) 
Proof: By recalling (T20l) and applying Lemmas [71 [8] of Appendices B, C to the closed-loop system 
F, it follows that 



d r Q = d r (\\F\\l) + 9d r (\\F\\i)/2 = 2 



H 
CP 



QB 




+ 29 



H 2 + Q$ + 
C(P# + $) 







which, in view of the notations (l42l) - (l44l) . implies (pd]) . ■ 
The Gramians P, Q of the closed-loop system and related matrices (that is, H, $, \P, T, f2, R) inherit 
the four (n x ra)-block structure of the matrix A in (l34l). The blocks are numbered as follows: 



A:- 



A n An 
•A21 A22 



-2n-> 



2» 



^4i. 
A 2 . 



^ 71 



(46) 



In this notation, the (-) n blocks are associated with the controller state, and the (-) 2 2 blocks pertain to 
the plant state. 

Lemma 4: The Frechet derivatives of the quadro-quartic functional Q of the closed-loop system (l34l) 
with respect to the controller matrices a, b, c, assembled into the matrix 7 in (1391 , are computed as 



<9 7 Q 



d c Q 



Rxi R12C2 + Q\,BD 21 



P> 2 R21 + Pi 2 CT.i 
where the matrices T, f2, R are defined by (I42l)-(l44l). 







(47) 
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Proof: Since Q is a composite function of a, b, c which enter this functional through the matrices 
A, B, C of the closed-loop system F, the chain rule yields 



<9 7 q = («9 7 r)t(d r Q) = n^^r^Qr 



(48) 



Here, denotes the adjoint of a linear operator in the sense of the Frobenius inner product of matrices, 



and n 



n,p2,m2 



is the orthogonal projection onto the subspace T 



n,P2,m 2 



defined by (|37H3H1>. Indeed, the 

first variation of the affine map T, defined by ([39]), is 5T = T 1 (Sj)T 2 . Hence, 5Q = Tr(<9 r Q5r T ) = 
Tr(a r Q(r 1 (5 7 )r 2 ) T ) = T¥(rf<9rQrj6 T T ) = Ti(U ntP2im2 (TjdrQTj)5-f T ), which establishes ©. 
Substitution of the matrices r\ and T 2 from (|40l) into the right-hand side of ( 1481) yields 

<9 7 Q 



n 



n,P2,m 2 



In 









d A Q d B Q 
d c Q 



In 






C 2 T 



ii 



5 2 T (^Q) 21 + Z)T(a c Q) 1 



(^Q) 12 C^ + (9 B Q) lJ D 




21 



Here, in view of d4T 



0aQ = 2i2, (0 B Q)i = 2^1.6, (<9 c Q)i = 2CT. 1; 



(49) 



(50) 



and the block numbering (|46l ) is used. The assertion (1471) of the lemma now follows from ( 1491 and (|50l) . 



Necessary conditions for optimality in the class of controllers (1321) for the LQQG problem (1351) are 
now obtained by equating the blocks of the matrix <9 7 Q in (14~7l) to zero: 



R\ 2 C 2 + Qi,BD 21 
BJR21 + -Dj^CT.i 



0. 
0. 

= 0. 



(51) 
(52) 
(53) 



VIII. Observation-state and state-feedback matrices 



Lemma 5: Suppose the matrix D 2 \ is of full row rank, and D\ 2 is of full column rank. Also, let 
(l32l) be a stabilizing controller with a minimal state-space realization. Then the top-left blocks of the 
matrices P, Q from (l36l) and T, Cl from (l42l) . (1431) are all positive definite: 



Pn^O, Qu^O, T n ^0, n n y0. (54) 
Proof: Since 6*^0, and the matrices Pff = PQP, HQ = QPQ, associated with the Gramians 
P, Q, and the Schattenians ^ from (1431) are all positive semi-definite, then (l42l) and (1431) imply that 
T y P and f2 )>= Q. Hence, the same ordering holds for the top-left blocks of these matrices: Tn )p Pn 
and f2 n y Qii. Therefore, the last two relations in (1541 ) will follow from the first two. We will now 
prove that Pn y under the assumptions that D 2 \ is of full row rank and (a, b) is controllable. Indeed, 
Pu is the covariance matrix of the controller state: 



Pn = cov(6 



1 

2^ 



g(uj)A(ui)g(ui)*du}, g(cu) := (iul n — a) b 



(55) 



where A (a;) : 
h(u) := D 21 



h(uj)h(u)* is the spectral density associated with the observation signal Y from dSTT) . with 
[0 C 2 ] {iul 2n - Aj^B. From lim^^ A(u) = D 21 D 2l , it follows that if D 21 is of full 
row rank, then A (a;) y for all sufficiently large to, say \cu\ > cu . Now, if P u is singular, then v T P n v = 
for some nonzero v G W 1 . In this case, (l55l) yields = u T Pnf ^ (27r) -1 Jj w i >Wo ||fi'( w )*^|lA( w )dw, 
which, in view of A (a;) y over the high frequency range, implies that v T g(co) = for all |o;| > uq. 
Hence, by considering the first n terms of the Laurent series v T g(oo) = Ylt=i u T a fc-1 &/ (ioo) k at infinity 
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[8, Lemma 2.3 on pp. 16-17], it follows that the rank of the matrix [b \ . . . | a n ~ l b] is less than n, and 
the pair (a, b) is not controllable. Thus, the full row rank of D 2 i and the controllability of (a, b) indeed 
ensure Pu y 0. By duality, a similar reasoning shows that the observability of (a, c) and the full column 
rank condition on D 12 imply Q n y 0. ■ 
Theorem 2: Suppose the matrix D 2 \ is of full row rank, and D\ 2 is of full column rank. Then the 
matrices b and c of an optimal controller (1321) in the LQQG problem (1351 ) with a minimal state-space 
realization satisfy 

b = -^(RnCj + fi 12J B 1J D 2 T 1 )( J D 2lJ D 2 T 1 )~ 1 , (56) 
c = -(Dj 2 D 12 )-\BjR 21 + D£CiT 21 )Tli\ (57) 

where the matrices T, f2, R are defined by (l42l)-([44l). 

Proof: Substitution of the matrices B and C from (|34|) into (l52l) and (l53l) brings these equations 
to the form 

R l2 C^ + (QnbD 21 + n 12 Bx)D^ = 0, (58) 
Bji^x + Dj 2 (D 12 cT 11 + dT 21 ) = 0. (59) 

By Lemma [5l the matrices Tn and fin are nonsingular. Therefore, left multiplication of both sides of 
(|58|) by fi n x and right multiplication by (/J^i-Dji) -1 yields (l56l) . Similarly, right multiplication of both 
sides of d591) by T n x and left multiplication by (D^D^)" 1 yields ([57]). ■ 
Under the assumptions of Theorem [2l the modified set of equations for the state-space realization 
matrices of an optimal controller in the LQQG problem (1351) is formed by the algebraic Lyapunov 
equations (l36l) . (l45l) and by the algebraic equations (IBTT) . (l56l) . (1571) . In the case 6* = 0, these equations 
can be shown to yield the two independent Riccati equations for the standard LQG controller. 



IX. HOMOTOPY METHOD 

With the matrix 7 from (|39l , we associate a linear subspace of r n p2 m2 by 

: r e E nxn | . (60) 

This is the tangent space generated by the group of transformations (a,b,c) (craa -1 , ab, ca^ 1 ) 
(where a E M nxn are arbitrary nonsingular matrices), which leave the transfer function of the controller 
(l32l) . and hence, the input-output operator of the closed-loop system (l39l) . unchanged. The matrix <9 7 Q, 
associated with the controller K, belongs to the orthogonal complement T(7) ± of ¥(7) to r np2jm2 in 
the sense of the Frobenius inner product. We say that the controller delivers a strong local minimum to 
the quadro-quartic functional Q in (|35T) if, in addition to the equality <9 7 Q = 0, it also makes the second 
order Frechet derivative d^Q = + 6 ) c?^(||F||4)/2 positive definite on the subspace ¥(7)-*-. 

Now, suppose there exists a smooth map ^ 9 1— > 7*(0) G r n p2 m2 such that y*(0) is a strong local 
minimum of the quadro-quartic functional Qq of the closed-loop system F in the sense above, so that 
SyQfll = 0. By differentiating the last equality with respect to 9, it follows that 

^Q fl ( 7 i) + a y (||F||*)/2 = 0. (61) 

Here, yi(9) := dey*{9) and use is made of the identity doQe = ||F|||/2 which follows from d20l) and, in 
view of the interchangeability of the derivatives in 9 and 7, implies that ded y Qg = d 7 (\\F\\f) /2 G T(7) ± . 
Since the matrix 7*(6 ) ) is defined up to the orbit of the transformation group, then 7*(6>) := cV}*(#) 
is defined modulo the subspace T(y*(8)) from (l60l) . Therefore, (loTI) . which is a linear equation with 
respect to ~fl(9), can be restricted to the subspace T^^^)) 1 . As long as 7*(6 ) ) is a strong local minimum 



T( 7 ) 



ra — ar rb 
-cr 
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of Qe, so that the self-adjoint operator <9 2 Q is positive definite (and hence, invertible) on T(7*(6')) ± , 
the equation (loTI) is equivalent to 

iM = -L- 1 (^(||^||l))/2, (62) 

where L is the restriction of <9 2 Q to the subspace T(7) ± . The equation (l62l) is an ODE, with 9 ^ 
playing the role of fictitious time. The initial value 7*(0) is provided by the state-space realization triple 
of the standard LQG controller. The computation of an LQQG controller for 9 > can be carried out 
by numerically integrating the homotopy ODE (|62l) initialized at 7*(0). The operator L involves Frechet 
differentiation of solutions of algebraic Lyapunov equations with respect to their coefficients, and the 
inverse L _1 can be computed by using the vectorization of matrices [10]. The state-space formulae of the 
homotopy algorithm and other details of its implementation will be reported in subsequent publications. 
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Appendix 

A. Covariance of squared norms of Gaussian random vectors 

Lemma 6: Let £ and 77 be jointly Gaussian random vectors with zero mean. Then the covariance of 
their squared Euclidean norms is expressed in terms of the Frobenius norm of their cross-covariance 
matrix by 

cov(|er,|r / | 2 ) = 2||cov(e,r / )|| 2 . (A.l) 
Proof: By applying the representation [7] for the mixed moments of Gaussian random variables in 
terms of their covariances to the entries of the vectors £ and 77, it follows that E(£ 2 ?7 2 ) = E(£j£j)E(?7j?7j) + 
+ = E(£ 2 )E(r/ 2 ) + 2(cov(6,r /i )) 2 . Therefore, 

E(|£| 2 M 2 ) = £ = E(|£| 2 )E(M 2 ) + 2 5>ovfe, (A.l) 

i,3 i,3 

where the rightmost sum is ||cov(£, 77) || 2 . The relation (|A. 1 b is now obtained by substituting (IA.2I ) into 
cov(|£| 2 , |?7| 2 ) := E(|£| 2 |?7| 2 ) - E(|£| 2 )E(|?7| 2 ). Note that (EAlT) can also be established by using [9, 
Lemma 6.2]. ■ 
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B. State space formula for Frechet derivative of Hi-norm 

Lemma 7: The Frechet derivative of the squared T^-norm E : 
Hurwitz, is computed as 

QB 



\F\\l of the system d22j), with A 



r : = 



.4 
C 



D 




H 

CP 

Here, the matrix H is associated by (1241) with the Gramians P, Q from (1231) . 

Proof: The Frechet derivative drE inherits the block structure of the matrix T: 

~d A E d B E] 
d c E 



(B.l) 



drE 



(B.2) 



We will now compute the blocks of this matrix. To calculate 8aE, let B and C be fixed. Then the first 
variation of E with respect to A is 5E = Ti{C T C5P) = -Tr((A T Q + QA)5P) = -Tt{Q{A5P + 
(SP)A T )) = Tr(Q((5A)P + P5A T )) = 2Tr(if <L4 T ), which implies that 



d A E = 2H. 



(B.3) 



Here, use has also been made of the first variation of the Lyapunov equation for P with constant B 
which yields ASP + (5P)A T + (SA)P + P5A T = 0. To compute d B E, we fix A and C. Then the 
observability Gramian Q, which is a function of A and C, is also constant, and the first variation of E 
with respect to B is 5E = Tr(Q5(BB T )) = Tr{Q{(SB)B T + B5B T )) = 2Tr(QB5B T ), and hence, 



d B E = 2QB. 



(B.4) 



The derivative dcE is calculated by a similar reasoning. Assuming A and B (and so also the 
controllability Gramian P) to be fixed, the first variation of E with respect to C is SE = Tr (PS(C T C)) = 
Tr(P((5C T )C + C T 5C)) = 2Tr(CP5C T ), which implies that 



d c E = 2CP. 

Substitution of (|BJ1)-(|B31) into (1ET21) yields (iBTl) . 

C. Frechet differentiation of quartic norm in state space 
Lemma 8: The Frechet derivative of the fourth power iV := 



(B.5) 



| FH! of the quartic norm of the system 



221) . with A Hurwitz, is computed as 

d r N = 4 



(HQ + V)B 




(C.l) 



H 2 + Q$ + #P 
C(PH+ $) 

Here, the matrix if is associated by (1241) with the Gramians P, Q from (|23l) , and $, \^ are the 
Schattenians from (1271) . 

Proof: We will compute the Frechet derivative of N by using the representation 



N = 2E 1= 2E; 



2, 



£i:= 



l-^illi 



Fo 



II^IIS, (C2) 

of the "H 4 -norm from Lemma [2] in terms of the squared "H 2 -norms of the subsidiary systems F\ := 
(A, B, B T Q) and F 2 := (A,PC T ,C) as composite functions of the matrices A, B, C. Since the 
controllability and observability Gramians of F 1 are P and ^, and the controllability and observability 
Gramians of F 2 are $ and Q, then application of Lemma [7] from Appendix B to the systems Fi and 
F 2 yields 

P T i/ 

c$ o 



dr 2 E 2 





a p" 


r i: = 


p T g o 




a pc t= 


r 2 := 


c 



(C.3) 
(C.4) 
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Suppose the matrices A and C are fixed and hence, so also is Q. Then (IC.3I ) implies that the first 
variation of E\ with respect to B is 

8E 1 = 2Tr(^ B5B T ) + 2Tr(5 T H5(B T Q) T ) 
= 2Tr((tf£ + QH T B)5B T ) 

= 2Tr((^ + HQ) B 5 B T ), (C.5) 

where the identity QH T = QPQ = HQ has also been used. From (IC.2I) and (IC.5I) . it follows that 

d B N = A(HQ + V)B. (C.6) 

Suppose the matrices A and 1? are fixed and hence, so also is P. Then (IC.4I) implies that the first 
variation of i? 2 with respect to C is 

5E 2 = 2Ti(HC T 5(PC T ) T ) + 2Tr(C<MC T ) 
= 2Tr((C$ + CH T P)5C T ) 

= 2Tr(C($ + PH)5C T ), (C.7) 
where the identity H T P = PQP = PH has also been used. From (IC.2I) and (IC.7I) . it follows that 

,9 c jV = 4C(P# + $). (C.8) 
Now, let B and C be constant. Then, in view of (|C.3I) . the variation of with respect to A is 

&E7i = 2Tr(^PM T ) + 2Tr (B T H5(B T Q) T ) = 2Ti(^ P5A T ) + 2Tr(iJ T BB T 5Q). (C.9) 

The first variation of the Lyapunov equation for Q in (|23T) with C constant yields A T 5Q + (5Q)A + 
(5A) T Q + Q5A = 0. Therefore, 

Tr((5Q)BB T H) = -Tr((5Q)(AP + PA T )H) 

= -Tt{(5Q)APH) - Tt{(5Q)PA t H) 

= Tr((A T 5Q + (5A) T Q + Q5A)PH) - Ti{{5Q)PA T H) 

= 2Tr(H 2 5A T ) + Ti(P(HA T - A T H)5Q) 

= 2Tt(H 2 5A t ) + Tr(P(C T CP - QBB T )5Q) 

= 2Ti(H 2 5A T ) - Tr((A$ + $A T )5Q) - Tr(H T BB T 5Q) 

= 2Tr((H 2 + Q$)5A T ) - Ti(H T BB T 5Q) 

= Tr((H 2 + Q$)5A T ). (CIO) 



Here, we have also used the definition of the controllability Schattenian $ in (1271) . and the identity 
HA T - A T H = C T CP - QBB T which is obtained from (f23]) and (fjl as = Q{AP + PA T + BB T ) - 
(A T Q + QA + C T C)P = HA T - A T H + QBB^ - C T CP. Substitution of (OOl) into CU> yields 
8E X = 2Tr((F 2 + Q$ + ^P)5A T ), which, in view of CJ), implies that 

d A N = 4(H 2 + Q<5> + ^P). (C.ll) 

The representation (IC.ll) now follows from (IC.6I) . (IC.8I) and (IC.11I) . ■ 
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